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* NOTICES * 

JPO and INPIT are not responsible for any 
damages caused by the use of this translation. 

1 .This document has been translated by computer. So the translation may not reflect the 
original precisely. 

2.**** shows the word which can not be translated. 
3. In the drawings, any words are not translated. 



DETAILED DESCRIPTION 

[Detailed Description of the Invention] 
[0001] 

[Field of the lnvention]This invention combines several pictures from which a light exposure 
differs, and relates to the image compositing device which expands and outputs the dynamic 
range of a picture signal. 
[0002] 

[Description of the Prior Art]ln the imaging device which has a solid state image pickup device, 
since the dynamic range of a solid state image pickup device is narrow compared with a silver 
halide film, when photoing a photographic subject, for example under a bright background, 
black crushing of a photographic subject and a white jump of a background arise. In order to 
prevent such a phenomenon, the picture of correct exposure is photoed to a photographic 
subject and each background, and the method of combining several pictures from which these 
light exposures differ is proposed. 

[0003]The key map of the method of combining two pictures from which a light exposure 
differs, and acquiring the large picture of a dynamic range is shown in drawing 20. In drawing 
20, the long exposure picture shown in (A) of drawing 20 has exposure in the person or the 
tree, and assumes that the background is white-flying. As for the short time exposure picture 
shown in (B) of drawing 20, exposure serves as a backdrop, and it assumes that the person is 
doing black crushing. Composition of these pictures can be performed, for example by the 
following methods. First, a suitable threshold is set up to a long exposure picture, and it is 
made for the pixel value of the field beyond a threshold to correspond with the field of a white 
jump. Subsequently, a short time exposure picture corresponds about the field beyond said 
threshold (homotopic). A picture is stuck and combined. Thus, by compounding, as shown in 
(C) of drawing 20, a background and a person can acquire the large picture of a dynamic 
range with exposure. 
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[0004] However, when the photographic subject is moving, for example since the 
photographing timing of each picture has shifted when two or more pictures from which a light 
exposure differs with one image sensor are photoed, or blurring occurs, a position gap arises 
between pictures. Thus, if two or more pictures which have produced the position gap are 
combined, as shown in drawing 21, a photographic subject will be an unnatural picture, such 
as being doubled. 

[0005]The proposal is made by JP, 31 10797, B as a method of amending a position gap of such 
a picture conventionally. Next, the conventional correcting method currently indicated by 
above-mentioned JP, 31 10797, B is explained. Drawing 22 is a key map showing the 
conventional correcting method. When the whole screen has shifted by blurring, by the 
conventional correcting method, the picture a and the picture b shift these pictures, and as 
shown in drawing 22, they are combining them so that the portion of the common area in 
drawing 22 may be obtained. 

[0006]Drawing 23 is a block diagram showing the example of composition of the image 
compositing device for enforcing the conventional correcting method. The memory control part 
205 controls the field memory 204a by the screen blur information detected by the motion 
detection 206 using the inputted image, and reads the picture equivalent to the common area 
in drawing 22 using it. After this common area picture is expanded to size equivalent to an 
inputted image in the magnification interpolation part 202a, it is stored in the field memory 
204b. Then, it is processed similarly and the picture outputted from the magnification 
interpolation part 202a and the picture of the previous frame read from the field memory 204b 
are combined with the adding machine 202b, and it is constituted so that the picture which 
amended blur may be acquired. In drawing 23, 201 is an A/D converter and 203 is a D/A 
converter. 
[0007] 

[Problem(s) to be Solved by the lnvention]As mentioned above, in the amendment art of the 
conventional proposal, the whole picture is shifted, a common area is obtained, blur 
amendment of the picture is performed, and it is a method suitable for amending blurring in 
which the whole picture blurs. However, photographic subject blur to which some photographic 
subjects move also exists in blur of a picture. The whole picture is carrying out blurring and the 
picture of two sheets shown in the left-hand side of drawing 24 assumes that the airplane has 
caused photographic subject blur in another direction with the direction of blurring. Therefore, if 
the whole screen is shifted and amended about a picture as shown in drawing 24 in the 
direction by which the airplane ran, it will be compounded as a picture from which other 
photographic subjects, such as a person and a tree, shifted. Thus, if the whole picture which 
carried out photographic subject blur is shifted and combined, it may not be compounded 
appropriately, but with the amendment art of the conventional proposal, consideration is not 
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made about this viewpoint. 

[0008]ln light of the above-mentioned circumstances, this invention is a thing. 
The purpose is to provide the image compositing device which simple processing detects any 
of photographic subject blur are produced, and the technique of picture composition is 
switched accommodative according to the kind of blur, and can acquire the large picture of the 
good dynamic range of image quality. 

[0009] 

[Means for Solving the Problem]ln order to solve an aforementioned problem, an invention 
concerning claim 1, In an imaging device which combines several pictures from which a light 
exposure differs, and acquires a large picture of a dynamic range, A blur kind judging means 
which judges what a difference between pictures to combine depends on photographic subject 
blur, or a thing to depend on blurring, Have a means for switching which switches an image 
synthesizing method according to a blur decision signal from this blur kind judging means, and 
said blur kind judging means, A motion vector primary detecting element which detects and 
outputs one motion vector as the whole picture between one picture and a picture of another 
side among said two or more pictures, It judges any of photographic subject blur and blurring 
pictures are, and it is with a blur judging means outputted as a blur decision signal, and an 
image compositing device consists of said motion vectors detected in this motion vector 
primary detecting element. 

[0010] 1st and 2nd embodiments correspond to an embodiment of the invention concerning 
above-mentioned claim 1. And to a motion vector primary detecting element of constituent 
features in claim 1, and a blur judging means. The motion vector primary detecting element 11 
in an embodiment (drawing 2) and the blur judging means 12 correspond, respectively, and 
what doubled these two blurs, it corresponds to a kind judging means, and the means for 
switching 13 and 13a are equivalent to a means for switching. 
[001 1]ln an image compositing device constituted in this way, a motion vector primary 
detecting element detects one motion vector as the whole picture, and a picture judges 
photographic subject blur or blurring by a blur judging means based on the motion vector. And 
since it is constituted so that a synthesizing method of a picture may be switched 
accommodative according to the decision result, image quality is good and can acquire a large 
picture of a dynamic range. 

[0012]ln an image compositing device concerning claim 1, an invention concerning claim 2 
said blur kind judging means, Having further a motion vector detection area specifying means 
which pinpoints a field which detects a motion vector, said motion vector primary detecting 
element detects a motion vector in a field which detects said motion vector. 
[0013]1st and 2nd embodiments correspond to an embodiment of the invention concerning this 
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claim 2. And the motion vector detection area specifying means 21 in an embodiment ( drawing 
3) is equivalent to a motion vector detection area specifying means of constituent features in 
claim 2. 

[0014]ln an image compositing device constituted in this way, a field which detects a motion 
vector is pinpointed and narrowed by a motion vector detection area specifying means, and it 
becomes possible to detect a motion vector by as little operation as possible. 
[0015]An invention which this invention requires for claim 3 is characterized by that an image 
compositing device concerning claim 2 comprises: 

A binarization part which said motion vector detection area specifying means carries out 
binarization of said one picture and the picture of said another side with a predetermined 
threshold respectively, and outputs a binary picture. 

A comparing element which performs comparison with said one binary picture and a binary 
picture of said another side, and outputs a discordance region as a field which detects said 
motion vector. 

[0016]1st and 2nd embodiments correspond to an embodiment of the invention concerning this 
claim 3. And the binarization parts 31 and 32 in an embodiment (drawing 6) correspond to a 
binarization part of constituent features in claim 3, and the comparing element 33 corresponds 
to a comparing element. And in a comparing element, as shown in drawing 7 a binary picture 
is compared by taking exclusive OR of the binary pictures a and b in the exclusive "or" circuit 
41, and binary picture discordance region information that a discordance region is set to "1" is 
outputted as a field which detects a motion vector. A motion vector detection area is equivalent 
to a portion displayed by a horizontal line by drawing i 4 

[0017]ln an image compositing device constituted in this way, since it constitutes so that a 
discordance region may be pinpointed by comparing the pictures which carried out 
binarization, a detection area of a motion vector can be pinpointed by simple processing. 
[0018]ln an image compositing device which an invention concerning claim 4 requires for claim 
3, said binarization part makes a threshold which carries out binarization of said one picture, 
and a threshold which carries out binarization of the picture of said another side a value 
proportional to a light exposure of each picture. 

[0019]1st and 2nd embodiments correspond to an embodiment of the invention concerning this 
claim 4. And threshold b= threshold ax (light exposure b / light exposure a) inputted into the 
threshold a and the binarization part 32 which are inputted into the binarization part 31 in an 
embodiment ( drawing 6 ) corresponds to a threshold of a binarization part of constituent 
features in claim 4. The threshold b in a formula (1) under explanation of an embodiment and a 
threshold inputted into said binarization part 32 are the same. 

[0020]lt is the same photographic subject (or background) also between pictures from which it 
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is proportioning a binarization threshold in a light exposure in an image compositing device 
constituted in this way, and a light exposure differs. Binarization can be carried out in the same 
value. Therefore, said photographic subject (or background) If existing coordinates have 
shifted, since values differ, on a binary picture, a discordance region of a binary picture is 
detectable as a field which has generated blur between pictures. 

[0021]ln an image compositing device concerning claim 3, an invention concerning claim 5 
said comparing element, It is constituted so that a signal which shows a field which scans said 
one binary picture and a binary picture of said another side in the predetermined direction, 
performs comparison between both images, restricts when the comparison result becomes 
inharmonious in succession in fixed numbers, and detects said motion vector may be 
outputted. 

[0022]1st and 2nd embodiments correspond to an embodiment of the invention concerning this 
claim 5 And the comparing element 33 shown in drawing 7 corresponds to a comparing 
element of constituent features in claim 5, and binary picture discordance region information in 
drawing 7 is equivalent to a binary picture comparison result. When a comparison result 
becomes inharmonious in succession in fixed numbers, as compared with the amount 
threshold of continuous detection, it is asking for what calculated binary picture discordance 
region information at the counter 42 by the comparator 43. And it has realized outputting an "it 
being limitation when comparison result becomes inharmonious in succession in fixed 
numbers" motion vector detection area by performing a change of the selector 44 using an 
output of said comparator 43. 

[0023]ln an image compositing device constituted in this way, disagreement generated by a 
noise etc. can be excepted and detection of a discordance region resulting from picture blur 
can be made into what has higher validity. 

[0024]An invention which this invention requires for claim 6 is characterized by that an image 
compositing device concerning claim 2 comprises: 

A block setting-out means by which said motion vector primary detecting element sets up 
specific 1 block in a field within said one [ corresponding to said motion vector detection area ] 
picture. 

A motion vector calculation part which calculates similarity with two or more blocks within a 
picture of said another side only to this specific 1 block, and outputs a vector to a block with 
the highest similarity as the above-mentioned motion vector. 

[0025] 1st and 2nd embodiments correspond to an embodiment of the invention concerning this 
claim 6. And the block setting-out means 22 in an embodiment (drawing 3) is equivalent to a 
block setting-out means of constituent features in claim 6, and the motion vector calculation 
part 23 corresponds to a motion vector calculation part. 
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[0026]ln an image compositing device constituted in this way, a block setting-out means sets 
only one block used as a standard in motion vector detection as one picture. And in a motion 
vector calculation part, similarity of this 1 block and two or more blocks within a picture of 
another side is calculated, and a vector to a high block of similarity is most outputted as a 
motion vector. Thus, a motion vector is detectable with a small operation amount by limiting a 
block which serves as a standard in calculation of a motion vector to one. 
[0027]ln an image compositing device which an invention concerning claim 7 requires for claim 
6, said motion vector calculation part is constituted so that said motion vector may be detected 
and outputted using the block matching method. 

[0028] 1st and 2nd embodiments correspond to an embodiment of the invention concerning this 
claim 7. In an image compositing device constituted in this way, a motion vector is detectable 
by an easy technique. 

[0029]ln an image compositing device which an invention concerning claim 8 requires for claim 
6, said motion vector primary detecting element has an effective judgment part which judges 
whether said motion vector is outputted to said blur judging means. 

[0030]1st and 2nd embodiments correspond to an embodiment of the invention concerning this 
claim 8. And the effective judgment part 24 in an embodiment (drawing 3) corresponds to an 
effective judgment part of constituent features in claim 8. 

[0031]ln an image compositing device constituted in this way, an effective judgment part 
checks the validity of a computed motion vector, and when validity is low, it carries out it as 
[ output / blur and / a motion vector / to a judging means ]. It can prevent outputting a 
meaningless motion vector by this, and accuracy of a blur judging can be raised. 
[0032]ln an image compositing device concerning claim 8, an invention concerning claim 9 
said effective judgment part, A black level primary detecting element which detects that a 
luminance value of said motion vector detection area of an inputted image is smaller [ than a 
predetermined value ] close to black, A white level primary detecting element which detects 
that a luminance value of said motion vector detection area of an inputted image is more 
greatly [ than a predetermined value ] close to white, It has a luminance difference primary 
detecting element which detects that a luminance value difference between each pixel of said 
motion vector detection area of an inputted image is smaller than a predetermined value, 
When all black level detection, white level detection, or luminance difference detection are not 
detected, it is constituted so that said motion vector may be made to output to said blur judging 
means. 

[0033] 1st and 2nd embodiments correspond to an embodiment of the invention concerning this 
claim 9. And the black crushing judgment part 52 in an embodiment (drawing 9) corresponds 
to a black level primary detecting element of constituent features in claim 9, and the white jump 
judgment part 51 corresponds to a white level primary detecting element. Composition which 
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consists of the maximum extraction part 53, the minimum extraction part 55, the subtractor 57, 
and the comparator 59 corresponds to a luminance difference primary detecting element. A 
luminance difference threshold in drawing 9 corresponds to a predetermined value in a 
luminance difference primary detecting element. And the maximum of the picture b and the 
minimum are calculated from the maximum extraction part 53 and the minimum extraction part 
55, respectively, difference is taken with the subtractor 57 and luminance difference is 
searched for. By investigating whether said luminance difference is below a luminance 
difference threshold by the comparator 59, it detects that a luminance value difference is 
smaller than a predetermined value. 

[0034]ln an image compositing device constituted in this way, when a picture in a motion 
vector detection area investigates whether it is a picture without luminance changes, such as 
black crushing, a white jump, and gray 1 color, and corresponds to them, a motion vector is not 
outputted as a picture unsuitable for motion vector detection. This becomes possible to raise 
accuracy of motion vector detection. 

[0035]ln an image compositing device concerning claim 8, an invention concerning claim 10 is 
constituted so that said motion vector may be made to output to said blur judging means, when 
said effective judgment part is [ a difference of the maximum of said similarity and the 
minimum ] beyond a predetermined range. 

[0036] 1st and 2nd embodiments correspond to an embodiment of the invention concerning this 
claim 10. And composition which consists of the maximum extraction part 54 in drawing 9 , the 
minimum extraction part 56, the subtractor 58, and the comparator 60 is equivalent to 
composition in claim 10 A similarity difference threshold value in drawing 9 corresponds to a 
predetermined range in claim 10. 

[0037]ln an image compositing device constituted in this way, When similarity of 1 block with 
one specific picture and two or more blocks of a picture of another side is measured 
respectively and there is no difference in them, Considering that block [ which ] is similar to the 
same extent, in this case, a vector to the highest block of similarity judges that validity is low as 
a motion vector, and does not output. This becomes possible to raise accuracy of motion 
vector detection. 

[0038]An invention which this invention requires for claim 1 1 is characterized by that an image 
compositing device concerning claim 3 comprises: 

A coordinate transformation means which shifts coordinates of a binary picture which requires 
said blur judging means for said one picture based on said motion vector to level and a 
perpendicular direction. 

A picture in which coordinate conversion was carried out by said coordinate transformation 
means. 

A calculus-of-finite-differences appearance means to input a binary picture concerning a 
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picture of said another side, and to output a discordance region between pictures as difference 
information. 

A blur decision signal creating means to which a picture judges either photographic subject 
blur or blurring, and outputs the decision signal from said difference information. 

[0039] 1st and 2nd embodiments correspond to an embodiment of the invention concerning this 
claim 1 1 . And the coordinate transformation means 61 in an embodiment (drawing 1 0) 
corresponds, the calculus-of-finite-differences appearance means 62 is equivalent to a 
calculus-of-finite-differences appearance means, it blurs in a blur decision signal creating 
means, and the decision signal creating means 63 is equivalent to a coordinate transformation 
means of constituent features in claim 1 1 . 

[0040]ln an image compositing device constituted in this way, the whole picture assumes a 
coordinate transformation means to have started blurring in the direction of a motion vector, 
and it shifts coordinates and changes one binary picture. In a calculus-of-finite-differences 
appearance part, it asks for a discordance region between this binary picture by which 
coordinate conversion was carried out, and a binary picture of another side, and it is judged 
from this result whether a picture was blurring. Thereby, it can be judged whether a picture has 
started blurring. 

[0041]ln an image compositing device which an invention concerning claim 12 requires for 
claim 11, said calculus-of-finite-differences appearance means is constituted so that a total of 
an inharmonious pixel between two binary pictures obtained by said comparing element may 
be outputted. 

[0042]1st and 2nd embodiments correspond to an embodiment of the invention concerning this 
claim 12. And composition shown in drawing 1 1 is equivalent to composition in claim 12. An 
inharmonious pixel between two binary pictures can be found by the exclusive "or" circuit 71 in 
drawing 1 1 , and a total of an inharmonious pixel is calculated by the counter 72 in drawing 1 1 . 
In an image compositing device constituted in this way, since it constitutes so that a total of an 
inharmonious pixel number between two binary pictures may be outputted, a blur judging can 
be performed as a size (blur degree) of a field of picture blur by it. 
[0043]lt is constituted so that it may judge with photographic subject blur, if an invention 
concerning claim 13 will judge said blur decision signal creating means in an image 
compositing device concerning claim 11 to be blurring if said difference information is below a 
predetermined threshold, and said difference information is beyond said threshold. 
[0044] 1st and 2nd embodiments correspond in an embodiment of the invention concerning this 
claim 13. And composition shown in drawing 13 is equivalent to composition in claim 13. In 
drawing 13, when a blur decision signal is "1" and it is blurring and "0", photographic subject 
blur is shown. Difference information can be found by the comparator [ in / in whether it is 



http://www4.ipdl.inpit.go.ip/cgi-bin/tran_web_cgi_eije?atw_u=http%3A%2F%2Fw 12/20/2008 



JP,2003-134385,A [DETAILED DESCRIPTION] 



Page 9 of 19 



below a predetermined threshold / drawing 13 ] 82. As for the comparator 82, difference 
information outputs "1" with "0" and below a difference threshold above a difference threshold. 
When difference information is below a difference threshold ("1"), a blur decision signal is set 
to "1" and is judged to be blurring, however, this time - an output of the motion vector 
existence judgment part 81 - "1" (it means that a motion vector was detected) it is -- it carries 
out. By constituting in this way, it can be judged as a picture from one motion vector whether a 
picture is carrying out blurring. 

[0045]ln an image compositing device which an invention concerning claim 14 requires for 
claim 1 1 , said blur decision signal creating means is characterized by a thing which judge with 
photographic subject blur and which is constituted so that it may carry out, when said motion 
vector is not detected. 

[0046]1st and 2nd embodiments correspond to an embodiment of the invention concerning this 
claim 14. And composition shown in drawing 13 is equivalent to composition in claim 14. In 
drawing 13, when a blur decision signal is "1" and it is blurring and "0", photographic subject 
blur is shown. Whether a motion vector was detected judges by the motion vector existence 
judgment part 81 in drawing 1 3. It means that a motion vector was not detected when an 
output of the motion vector existence judgment part 81 was "0", and when an output is "1 ", it 
means that a motion vector was detected. When an output of the motion vector existence 
judgment part 81 is "0", a blur decision signal is set to "0" and is judged to be photographic 
subject blur. By constituting in this way, a motion vector can judge with photographic subject 
blur uniformly about a picture whose detection is impossible. 

[0047]ln an image compositing device concerning claim 1, an invention concerning claim 15 
said means for switching, When said blur decision signal shows photographic subject blur, it is 
constituted to a picture acquired by combining said one picture and a picture of said another 
side so that a low pass filter may be applied to a picture signal corresponding to said motion 
vector detection area. 

[0048]1st and 2nd embodiments correspond to an embodiment of the invention concerning this 
claim 15. And composition shown in drawing 14 and drawing 18 is equivalent to composition in 
claim 15. Composition with said one picture and a picture of said another side is performed by 
the synthesizing means 92. The low pass filter 93 corresponds to a low pass filter in this claim. 
It is connected to the switch 94 and 95s2 when a blur decision signal shows photographic 
subject blur. Therefore, after combining the pictures a and b, a low pass filter is applied to an 
imaging range corresponding to a motion vector detection area. 

[0049]ln an image compositing device constituted in this way, since it is constituted so that a 
low pass filter may be applied to a portion of photographic subject blur, a portion of blur can be 
obscured and a large picture of a dynamic range is acquired by fitness of image quality. 
[0050]ln an image compositing device concerning claim 1, an invention concerning claim 16 
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said means for switching, When said blur decision signal shows blurring, they are level and the 
thing constituting so that a picture which shifted coordinates perpendicularly, and a picture of 
said another side may be combined about said one picture based on said motion vector. 
[0051 ]A 1st embodiment corresponds to an embodiment of the invention concerning this claim 

16. And composition shown in drawing 14 is equivalent to composition in claim 16. Processing 
which is level and shifts coordinates perpendicularly is performed in said one picture based on 
said motion vector by the coordinate transformation means 91 in drawing 14. When a blur 
decision signal shows blurring, both the switches 94 and 95 are connected to s1. Therefore, 
the picture b is transformed and said picture by which coordinate conversion was carried out, 
and the picture a are combined. 

[0052]ln an image compositing device constituted in this way, since it is constituted so that one 
picture may be shifted and combined in the direction which amends blurring, image quality is 
good and a large picture of a dynamic range is acquired. 

[0053]A binary picture where an invention concerning claim 17 carries out binarization of said 
two or more pictures with a predetermined threshold respectively, and obtains a binary picture 
in an image compositing device concerning claim 1 and which starts a picture of said another 
side, From a picture which was level and was shifted perpendicularly, search for coordinates of 
a binary picture which starts said one picture based on said motion vector, and a discordance 
region between pictures said means for switching, When said blur decision signal shows 
blurring, said one picture based on said motion vector Level and a picture which shifted 
coordinates perpendicularly, To a blurring correction picture acquired by combining a picture of 
said another side, it is constituted so that a low pass filter may be applied to a picture signal 
corresponding to said discordance region. 

[0054]A 2nd embodiment corresponds to an embodiment of the invention concerning this claim 

17. And in composition in claim 17, composition of drawing 6, drawing 10, drawing 16, and 
drawing 18 corresponds. A method of obtaining a binary picture in this claim is based on the 
binarization parts 31 and 32 in drawing 8 , and a picture which carried out binarization is stored 
in the memory 3 in drawing 18. A method of asking for a discordance region between pictures 
coordinates of one binary picture a binary picture of another side, and based on a motion 
vector from a picture which was level and was shifted perpendicularly, It is based on 
composition which consists of the coordinate transformation means 61 in drawing 10, and the 
calculus-of-finite-differences appearance means 62, and is equivalent to a field of "1" of a 
difference image in drawing 10 with a discordance region between pictures. A binary picture 
generated by the above and composition of drawing 6 is read from the memory 3, composition 
of drawing 10 generates a difference image, and it stores in the memory 3. A difference image 
in drawing 18 reads what is stored in the memory 3 by an above-mentioned method. When a 
blur decision signal shows blurring, the switches 94, 95, and 96 are connected to s1 , s2, and 
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s1, respectively. Therefore, the picture b is transformed, said picture by which coordinate 
conversion was carried out, and the picture a are combined, it ranks second, and a low pass 
filter is applied to an imaging range corresponding to a field of "1 " of said difference image. 
[0055]ln an image compositing device constituted in this way, the whole picture assumes that 
blurring is started in the direction of a motion vector, coordinates are shifted, one binary picture 
is changed, and a discordance region is detected as a field which amends blurring and where 
picture blur remains in addition. When judged with picture blur having been blurring, field (field 
where blur remains) further corresponding to [ based on a motion vector, shift coordinates, 
combine one picture, and ] said discordance region **** - by obscuring applying it, Blurring 
and photographic subject blur are amended simultaneously, and image quality is good and can 
acquire a large picture of a dynamic range. 
[0056] 

[Embodiment of the lnvention](A 1st embodiment), next a 1st embodiment of the image 
compositing device concerning this invention are described. Drawing 1 is a block diagram 
showing the basic constitution of the imaging device containing the image compositing device 
concerning a 1st embodiment. The CCD image sensor which 1 carries out photoelectric 
conversion of the object image, and is outputted as an electrical signal, A memory for the 
image processing device with which 2 performs color correction, edge enhancement, etc. of a 
picture, and 3 to store image data, The image compositing device with which 4 combines two 
or more pictures, CPU by which 5 controls each of these devices, and 6 are the buses for 
connecting other devices (an image processing device, an image compositing device, CPU) 
with a memory. 

[0057]The picture picturized by CCD1 is stored in the memory 3 via the bus 6. Several pictures 
from which a light exposure differs are beforehand photoed in the memory 3, and are stored in 
it. The image compositing device 4 inputs several pictures from which said light exposure 
stored in the memory 3 differs, judges the kind of blur, performs the compositing process 
according to the decision result, and stores it in the memory 3 as a blur correction picture. The 
amended picture is read from the memory 3 if needed, and is processed in the image 
processing device 2. 

[0058]Next, an image compositing device is explained in detail. In the following explanation, 
among several pictures from which said light exposure differs, one side is called the picture a 
and another side is called the picture b. Drawing 2 is a block lineblock diagram showing the 
example of composition of the image compositing device 4 in drawing 1 . In drawing 2 , 1 1 is a 
motion vector primary detecting element, and one motion vector is detected as the whole 
picture in this motion vector primary detecting element 1 1 using the pictures a and b read from 
the memory 3. In the case of motion vector detection, the binary pictures a and b which carried 
out binarization of the pictures a and b are generated, and it is stored in the memory 3. In the 



)://www4.ipdl.inpit.go.ip/cgi-bin/tran_web_cgi_eiie?atw_u=http%3A%2F%2Fwww4.i... 12/20/2008 



JP,2003-134385,A [DETAILED DESCRIPTION] 



Page 12 of 19 



blur judgment part 12, using said motion vector and the binary pictures a and b, it judges 
whether it is that blurring has occurred and outputs. In the means for switching 13, based on 
the result of said blurring judging, the switches 16 and 17 are switched suitably, either the 
blurring correction treating part 14 or the photographic subject blur correction processing 
section 15 is processed, and image composing is outputted. 

[0059]Next, the details of each members forming of the image compositing device shown in 
drawing 2 are explained. Drawing 3 is a block lineblock diagram showing the example of 
composition of the motion vector primary detecting element 1 1 in drawing 2 . The field which 
detects a motion vector is pinpointed in the motion vector detection area specifying means 21. 
As shown in drawing 4 , the portion from which the photographic subject has shifted among the 
pictures a and b is specified as a motion vector detection area. The information on the 
pinpointed motion vector detection area is once stored in the memory 3. It stores in the 
memory 3 also about the binary pictures a and b (after-mentioned) generated when a motion 
vector detection area is pinpointed. In the block setting-out means 22, the information on said 
motion vector detection area is read from the memory 3, and specific 1 block is set up as a 
standard block in motion vector detection. As shown in drawing 5 the block set up by the block 
setting-out means 22 is considered as a standard block, a search range is established in the 
circumference, and a motion vector is detected by the motion vector calculation part 23. A 
standard block and a search range are set up in a different picture. For example, when setting 
a standard block as the picture b, a search range is set as the picture a. 
[0060]A motion vector is computed only to the standard block set up by the block setting-out 
means 22, and is taken as one motion vector by making the result into the whole picture. As a 
method of computing a motion vector, the block matching method is used, for example. To the 
standard block set up by the block setting-out means 22, it asks for similarity with each block in 
the search range of a motion vector, and let the vector to the high block of similarity be a 
motion vector most, in the effective judgment part 24, the validity of whether the block set up 
by the block setting-out means 22 is appropriate as a standard block of motion vector detection 
and the motion vector computed by the motion vector calculation part is judged. When these 
are not appropriate, the motion vector calculation part 23 is controlled not to output the 
computed motion vector. 

[0061]Although the above-mentioned explanation showed what stored the information on a 
motion vector detection area in the memory 3, a memory for exclusive use may be provided 
between the motion vector detection area specifying means 21 and the block setting-out 
means 22. 

[0062]Drawing 6 is a block diagram showing the example of composition of the motion vector 
detection area specifying means 21 in drawing 3. Binarization of the pictures a and b is carried 
out by the binarization parts 31 and 32, respectively, and the binary pictures a and b are 
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generated. At this time, it sets up so that the binarization threshold of the pictures a and b may 
satisfy the relation of a following formula (1). 

Threshold b= threshold ax (light exposure b / light exposure a) (1) 

By setting up each threshold like a formula (1), if there is no gap among the pictures a and b, 
the same binary picture can completely be obtained. Therefore, the portion which has 
generated the gap conversely will differ [ binary pictures / a and b ] in a value. 
[0063]Said binary pictures a and b are compared by the comparing element 33, and a motion 
vector detection area is outputted as a binary picture whose field where values differ is "1." 
Drawing 7 is a block lineblock diagram showing the still more detailed example of composition 
of the comparing element 33. The binary picture discordance region information that a 
discordance region is set to "1" is generated by taking the exclusive OR of the binary pictures 
a and b in the exclusive "or" circuit 41. The number of times which "1" followed counts this 
binary picture discordance region information with the counter 42 which considers it as 
enabling, and only when that counted value exceeds the amount threshold of continuous 
detection beforehand set up by the comparator 43, "1" is outputted through the selector 44. By 
taking this composition, the disagreement produced under the influence of a noise etc. is 
removed. 

[0064]Drawing_8_js an explanatory view about the block setting method in the block setting-out 
means 22 in drawing 3 In the block setting-out means 22 specific 1 block in the motion vector 
detection area stored in the memory 3 is set up as a standard block of motion vector detection. 
In the example of a graphic display shown in drawing 8 , a picture is divided into block like 
shape and the information on a motion vector detection area is spirally searched toward the 
rim from the block of the center of a screen. If the block with which "1 " (discordance region) is 
contained first is discovered, the block of the picture b corresponding to the position of the 
block will be read from the memory 3, and it will output as a standard block. According to such 
a method, the block near the center of a picture if possible can be set up as a standard block, 
and a possibility that a motion vector can be discovered can be made high compared with the 
case where a standard block is taken at the end of a screen. 

[0065]Naturally, a block setting method is not limited to the above-mentioned technique. For 
example, a motion vector detection area may be searched sequentially from the screen upper 
left, and the block with which a discordance region is included first may be set up as a 
standard block. According to such a method, addressing at the time of reading the information 
on a motion vector detection area from the memory 3 can be performed by a simple method. 
The inharmonious consecutive number to a screen transverse direction may set up the block 
included to the greatest field as a standard block. A discordance region long in such a screen 
transverse direction has the high possibility of the gap by blurring generated in the screen 
lengthwise direction, and can make high a possibility that the motion vector by blurring will be 
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detectable. Naturally, the inharmonious consecutive number to a screen lengthwise direction 
may set up the block included to the greatest field as a standard block. It is also possible to set 
up a block, combining the above-mentioned method two or more. 
[0066] Drawing 9 is a block diagram showing the example of composition of the effective 
judgment part 24 in drawing 3 A white jump, black crushing, and a gray judgment are 
performed to the picture b. The white jump judgment part 51 will judge said field to be a white 
jump, if the total pixel value of the arbitrary area of the picture b is beyond a white threshold. 
The black crushing judgment part 52 will judge said field to be black crushing, if the total pixel 
value of the arbitrary area of the picture b is below a black threshold. The pixel value maximum 
and the minimum of an arbitrary area of the picture b are calculated from the maximum 
extraction part 53 and the minimum extraction part 55, respectively, and the difference value is 
computed with the subtractor 57. A difference value is compared with a luminance difference 
threshold by the comparator 59, and if it is below a threshold, it will judge with there being no 
luminance difference and it being gray in a field. 

[0067]The validity of a similarity lost-motion vector is judged. The maximum and the minimum 
are calculated by the maximum extraction part 54 and the minimum extraction part 56 from a 
set of the similarity outputted from the motion vector calculation part 23. The difference value is 
computed with the subtractor 58, and if it is below a threshold by the comparator 60 about the 
result as compared with a similarity difference threshold value, the vector to the maximum of a 
set of said similarity will be judged as there being no validity in considering it as a motion 
vector. 

[0068]When considering the arbitrary area of the described image b as the standard block set 
up by the block setting-out means 22 and a white jump, black crushing, or gray is judged, it 
can be judged that the set-up standard block is not suitable for motion vector detection. If the 
search range (picture a) in the motion vector calculation from a standard block is inputted into 
these judgment parts, it can be judged whether said search range is suitable for motion vector 
detection. 

[0069]A valid signal is generated combining each result of the white jump judging in drawing 9 , 
a black crushing judging, a gray judging, and a motion vector invalid judging, and it outputs to 
the block setting-out means 22 and the motion vector calculation part 23. Although any 
combination is possible, for example, when all white jumps, black crushing, and gray are not 
judged, a valid signal is outputted to the block setting-out means 22, and when judged with 
motion vector invalidity, a valid signal is outputted to the motion vector calculation part 23. 
[0070]Drawing 1 0 is a block diagram showing the example of composition of the blur judging 
means 12 in drawing 2. The coordinate transformation means 61 shifts the binary picture b 
read from the memory 3 to level based on a motion vector, and a perpendicular direction, and 
outputs it as binary picture b'. By the calculus-of-finite-differences appearance means 62, the 
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binary picture a and b' are compared and a difference image and difference information are 
outputted. Drawing 1 1 is a block diagram showing the still more detailed example of 
composition of the calculus-of-finite-differences appearance means 62. The exclusive OR of 
the binary picture a and b' is taken in the exclusive "or" circuit 71 , it asks for the discordance 
region between the binary picture a and b', and it is outputted as a difference image. The 
number of "1 " of a difference image is calculated with the counter 72, and the pixel number of 
the discordance region between the binary picture a and b' is outputted as difference 
information. The field which becomes inharmonious [ a difference image ] when the whole 
picture assumes that blurring is produced in the direction of a motion vector and two pictures 
are combined, as shown in drawing 12 is shown, and, as for difference information, the size of 
the discordance region is shown. In the blur decision signal creating means 63, from said 
difference information and a motion vector, blurring or photographic subject blur is judged, it 
blurs and a result is outputted as a decision signal. 

[0071]Drawing 13 is a block diagram showing the still more detailed example of composition of 
the blur decision signal creating means 63. Here, blurring and the case of "0" are considered 
for the case where a blur decision signal is "1" as photographic subject blur. The motion vector 
existence judgment part 81 outputs "0", when a motion vector calculation part 23 lost-motion 
vector is outputted and a motion vector is not outputted in "1" (i.e., when a motion vector is not 
detected). On the other hand, difference information is compared with a difference threshold by 
the comparator 82, in below a difference threshold, "1" is outputted and, in the above case, "0" 
is outputted. In AND83, the two above-mentioned binary signals take a logical product, blur, 
and are outputted as a decision signal. According to such composition, a motion vector is 
detected, and when difference information is below a threshold, a blur decision signal is set to 
"1" and is judged to be blurring. When difference information was beyond the threshold, or 
when a motion vector is not detected, a blur decision signal is set to "0" and is judged to be 
photographic subject blur. 

[0072]A difference threshold can be set up arbitrarily here. For example, if a certain specific 
value is set up, it can be considered that the case where it becomes below a specific value 
with the size of the discordance region at the time of regarding it as blurring and amending is 
blurring. It is good also considering the pixel number of the discordance region of the binary 
pictures a and b as a difference threshold. In this case, what is necessary is just to let the 
result of having added and outputted a means to calculate the pixel number of the discordance 
region of the binary pictures a and b to the motion vector detection area specifying means 21 
shown in drawing 3 be a difference threshold. Thus, when setting up the difference threshold, 
as a result of regarding it as blurring and amending, it can be considered that the case where 
discordance regions decrease in number before and after amendment is blurring. 
[0073]Drawing 14 is a block diagram showing the concrete example of composition of the 
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means for switching 13 in drawing 2 . The coordinate transformation means 91 generates the 
picture which was level and was shifted perpendicularly for the picture b based on a motion 
vector. The synthesizing means 92 is a synthesizing means which combines two pictures. The 
value of the binary picture a is "1" (or "0"). The pixel corresponding to a field is the picture a 
"0" (or "1") The pixel corresponding to a field is compounded using another picture b. The low 
pass filter 93 performs low pass filter processing only to the field in the inputted image 
corresponding to a motion vector detection area. The switches 94 and 95 are switches with 
which connection switches according to a blur decision signal. 

[0074]Next, a blur decision signal explains the picture composition operation about the cases 
of photographic subject blur and blurring. When a blur decision signal is photographic subject 
blur, it amends to the blur field of the compound picture, applying it. First, it connects with the 
switch 94s2 side, and the pictures a and b are combined by the synthesizing means 92. The 
picture of the field corresponding to a motion vector detection area blurred, and the picture 
acquired here was being combined, as shown in drawing 15. Low pass filter processing is 
performed to the field corresponding to a motion vector detection area to said combined 
picture. It connects with the switch 95s2 side, and stores in the memory 3 by using as image 
composing the picture to which said low pass filter was applied. The portion into which the 
picture has blurred can be made not conspicuous by this, and the large picture of a dynamic 
range can be acquired by the fitness of image quality. 

[0075]When a blur decision signal is blurring, the whole picture is shifted and combined in the 
direction of a motion vector. First, the picture which shifted the picture b in the direction of a 
motion vector by the coordinate transformation means 91 is generated. It connects with the 
switch 94s1 side, and the picture a and the picture which shifted said picture b in the direction 
of a motion vector are combined in the synthesizing means 92. It connects with the switch 
95s1, and a low pass filter is stored in the memory 3, without applying. As shown in drawing 
16, the picture acquired by this processing is the picture by which picture blur by blurring was 
amended, and can acquire the good and dynamic large picture of image quality. 
[0076]As mentioned above, whether according to this embodiment, the picture has produced 
blurring, or photographic subject blur is produced. Since it judges using only the operation 
between one a motion vector and a binary picture as a picture and the method of picture 
composition is appropriately switched according to the kind of blur, it can blur in simple 
processing, and can judge, and the large picture of a dynamic range can be acquired by the 
fitness of image quality. 

[0077](A 2nd embodiment), next a 2nd embodiment of this invention are described. Drawing 
17 is a block diagram showing the composition of the image compositing device concerning 
this embodiment. Since the composition of those other than means-for-switching 13a is the 
same as that of a 1st embodiment, it omits explanation. In the means for switching 13a in this 
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embodiment, the switches 16-18 are switched suitably, either the blurring correction treating 
part 14 or the photographic subject blur correction processing section 15 and both processings 
are performed, and image composing is outputted. 

[0078]Drawing 18 is a block diagram showing the concrete example of composition of the 
means for switching 13a in this embodiment. Since the coordinate transformation means 91, 
the synthesizing means 92, the low pass filter 93, and the switches 94 and 95 are the same as 
that of the means for switching 13 in a 1st embodiment shown in drawing 14, they omit 
explanation. The switch 96 is a switch which switches the field to which a low pass filter is 
applied with a motion vector detection area and a difference image according to a blur decision 
signal. 

[0079]Next, a blur decision signal explains the picture composition operation about the cases 
of photographic subject blur and blurring. Since a motion vector detection area is chosen as a 
field which connects with the switch 96s2 side and to which a low pass filter is applied when a 
blur decision signal is photographic subject blur, the same processing as a 1st embodiment is 
performed. When a blur decision signal is blurring, it is the same as that of a 1st embodiment 
until it shifts and combines the whole picture in the direction of a motion vector. That is, the 
picture which shifted the picture b in the direction of a motion vector by the coordinate 
transformation means 91 is generated, it connects with the switch 94s1 side, and the picture a 
and the picture which shifted said picture b in the direction of a motion vector are combined by 
the synthesizing means 92. 

[0080]Subsequently, it connects with the switch 96s1 side, and low pass filter processing is 
performed to the field corresponding to a difference image. As drawing 12 showed, a 
difference image is a discordance region of the binary picture a and the picture which shifted 
the binary picture b in the direction of a motion vector, and it is in agreement with the blur 
generating region of the correction picture by blurring. Therefore, a low pass filter will start only 
the blur field which remained after amendment by blurring. Finally it connects with the switch 
95s2 side, and stores in the memory 3 by using as image composing the picture to which said 
low pass filter was applied. 

[0081]Both drawing 19 shows the example which combined the picture which blurring and 
photographic subject blur have generated. It becomes impossible for the blur portion of an 
airplane to be conspicuous by applying a low pass filter with a described method, and after 
amendment of blurring can acquire the large picture of a dynamic range by the fitness of image 
quality, although blur remains in the portion of an airplane. 

[0082]As mentioned above, when the picture has started blurring, by shifting and combining a 
picture, blurring is amended, and a low pass filter is further applied to the blur portion which 
remained after amendment, and it is not conspicuous and is possible [ blur ] according to this 
embodiment. Therefore, the large picture of a dynamic range can be acquired by the fitness of 
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image quality also to the picture in which blurring and photographic subject blur were 

intermingled. 

[0083] 

[Effect of the lnvention]As it explained based on the embodiment above, according to the 
invention concerning claim 1. Since one motion vector is detected as the whole picture, a 
picture judges photographic subject blur or blurring based on the motion vector and he is trying 
to switch the synthesizing method of a picture accommodative according to the decision result, 
image quality is good and can acquire the large picture of a dynamic range. According to the 
invention concerning claim 2, since the field which detects a motion vector is pinpointed and 
narrowed, a motion vector is detectable by little operation. According to the invention 
concerning claim 3, by comparing the pictures which carried out binarization, a discordance 
region can be pinpointed and the detection area of a motion vector can be pinpointed by 
simple processing. According to the invention concerning claim 4, it becomes possible to 
detect the discordance region of a binary picture as a field which has generated blur between 
pictures. According to the invention concerning claim 5, the discordance region which a noise 
etc. generate can be excepted and detection of the discordance region resulting from picture 
blur can be made into what has higher validity. According to the invention concerning claim 6, 
since he is trying to limit the block which serves as a standard in calculation of a motion vector 
to one, a motion vector is detectable with a small operation amount. According to the invention 
concerning claim 7, a motion vector is easily detectable by using the block matching method. 
According to the invention concerning claim 8, a meaningless motion vector can be prevented 
from being outputted and the accuracy of a blur judging can be raised. 
[0084]According to the invention concerning claim 9, it investigates whether the picture in a 
motion vector detection area is a picture without a luminance change, and since he is trying 
not to output a motion vector as a picture unsuitable for detection of a motion vector when it 
corresponds, the detecting accuracy of a motion vector can be raised. According to the 
invention concerning claim 1 0, since it judges that the validity of the computed motion vector is 
low and he is trying not to output when there is no difference between similarity with two or 
more blocks, the detecting accuracy of a motion vector can be raised. According to the 
invention concerning claim 1 1 , since he is trying for while for coordinate conversion to have 
been carried out to judge blurring of a picture in quest of the discordance region between a 
binary picture and the binary picture of another side, it can be judged correctly whether the 
picture has started blurring. Since he is trying to output the total of the inharmonious pixel 
number between two binary pictures according to the invention concerning claim 12, it can 
judge by blurring as a size of the field of picture blur by it. According to the invention 
concerning claim 13, it can be judged whether the picture is carrying out blurring from one 
motion vector as a picture. According to the invention concerning claim 14, about the picture 
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which a motion vector cannot detect, it can judge with photographic subject blur uniformly. 
Since he is trying to apply a low pass filter to the portion of photographic subject blur according 
to the invention concerning claim 15, a blur portion can be obscured, image quality is good and 
the large picture of a dynamic range is acquired. According to the invention concerning claim 
16, since one picture is shifted in the direction which amends blurring and he is trying to 
compound in it, the large picture of the good dynamic range of image quality is acquired. Since 
he is trying to obscure according to the invention concerning claim 17, shifting coordinates, 
and combining one picture based on a motion vector, and also applying a low pass filter to the 
field to which blur corresponding to a discordance region remains, Blurring and photographic 
subject blur can be amended simultaneously and the large picture of the good dynamic range 
of image quality can be acquired. 



[Translation done.] 
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